Video commonly is processed to create special effects by tracking an object in the video. For example, the video may be processed using the information about the tracked object to remove movement of the object (an effect called stabilization). An object also may be added to the video that has a position or movement related to the tracked object (an effect called compositing). An object that has a relationship with the tracked object also may be removed from the video (such as an effect called wire removal).
Video may be stabilized before compositing, wire removal or other effects are performed where the movement of objects in the video may impact the quality of the effect. The movement of the objects may be reintroduced into the video after the effect is performed by inverting the stabilization effect.
Creating an effect using tracking and stabilization involves multiple operations to be performed on each image to produce the effect. These operations generally prevent such effects from being rendered in real time. The inability to view the effect in real time increases the cost incurred for each change made to the effect, which in turn limits the creativity of the editor.
Real time effects are provided by using tracking information to define values for parameters of an effect. The tracking information also may be combined with parameters for various effects that operate on the position, rotation, scale, skew and perspective of objects in an image. The values defined for the parameters of an effect may be represented in a transformation matrix used in a real time digital video effects processor.
Accordingly, in one aspect, effects may be generated in real time by key frames on video data comprising a sequence of images using tracking information. For each image that is not a key frame, one or more values for parameters are interpolated for the effect using the one or more values that are specified at the key frames. The tracking information is combined with the one or more interpolated values and the effect is performed on each image in the sequence of images in real time using the combined tracking information and one or more interpolated values.